Skip to content

YARN-10747. Bump YARN CSI protobuf version to 3.7.1#2946

Merged
aajisaka merged 3 commits intoapache:trunkfrom
smengcl:YARN-10747
Apr 28, 2021
Merged

YARN-10747. Bump YARN CSI protobuf version to 3.7.1#2946
aajisaka merged 3 commits intoapache:trunkfrom
smengcl:YARN-10747

Conversation

@smengcl
Copy link
Contributor

@smengcl smengcl commented Apr 22, 2021

@smengcl smengcl self-assigned this Apr 22, 2021
@smengcl smengcl requested a review from sunilgovind April 22, 2021 05:10
@aajisaka
Copy link
Member

aajisaka commented Apr 22, 2021

I think hadoop.protobuf.version in hadoop-project/pom.xml can be used instead of defining the version in hadoop-yarn-csi module.

@hadoop-yetus

This comment has been minimized.

Sign in to view
@smengcl
Copy link
Contributor Author

smengcl commented Apr 22, 2021

I think hadoop.protobuf.version in hadoop-project/pom.xml can be used instead of defining the version in hadoop-yarn-csi module.

Good point! Done.

@hadoop-yetus

This comment has been minimized.

Sign in to view
@smengcl
Copy link
Contributor Author

smengcl commented Apr 22, 2021

This is weird. The second commit essentially translate into the same thing as the first one, but now it gets mvninstall -1 for Dependency convergence error:

[WARNING] 
Dependency convergence error for com.google.protobuf:protobuf-java:3.7.1 paths to dependency are:
+-org.apache.hadoop:hadoop-yarn-csi:3.4.0-SNAPSHOT
  +-com.google.protobuf:protobuf-java:3.7.1
and
+-org.apache.hadoop:hadoop-yarn-csi:3.4.0-SNAPSHOT
  +-io.grpc:grpc-protobuf:1.26.0
    +-com.google.protobuf:protobuf-java:2.5.0
and
+-org.apache.hadoop:hadoop-yarn-csi:3.4.0-SNAPSHOT
  +-org.apache.hadoop:hadoop-common:3.4.0-SNAPSHOT
    +-com.google.protobuf:protobuf-java:2.5.0
and
+-org.apache.hadoop:hadoop-yarn-csi:3.4.0-SNAPSHOT
  +-org.apache.hadoop:hadoop-common:3.4.0-SNAPSHOT
    +-com.google.protobuf:protobuf-java:2.5.0
and
+-org.apache.hadoop:hadoop-yarn-csi:3.4.0-SNAPSHOT
  +-org.apache.hadoop:hadoop-yarn-common:3.4.0-SNAPSHOT
    +-com.google.protobuf:protobuf-java:2.5.0
and
+-org.apache.hadoop:hadoop-yarn-csi:3.4.0-SNAPSHOT
  +-org.apache.hadoop:hadoop-yarn-api:3.4.0-SNAPSHOT
    +-com.google.protobuf:protobuf-java:2.5.0

[WARNING] Rule 0: org.apache.maven.plugins.enforcer.DependencyConvergence failed with message:
Failed while enforcing releasability. See above detailed error message.

@aajisaka
Copy link
Member

aajisaka commented Apr 26, 2021

Thanks @smengcl for the update.

Before: While checking the dependency, hadoop.version overrides the protobuf-java versions in hadoop-common and other modules. Therefore the dependency convergence error is missed.

After: The dependency check is run as expected, and it fails.

Reading the discussion in YARN-9057, YARN CSI does not contain third-party dependencies including protocol buffer 2.x jars. Therefore we can exclude the protocol buffer 2.x dependency from hadoop-common and other modules to fix the error.

@smengcl
Exclude protobuf-java 2.5.0 from yarn-csi.
fd85854 
Change-Id: Ib2a4bf775a5804641a91c6f9a486772c9d87c17e
@smengcl
Copy link
Contributor Author

smengcl commented Apr 26, 2021

Thanks @aajisaka . I have excluded protobuf-java 2.5.0 from yarn-csi dependencies that introduced it.

@hadoop-yetus

This comment has been minimized.

Sign in to view
@aajisaka aajisaka merged commit 5b09ba7 into apache:trunk Apr 28, 2021
@aajisaka
Copy link
Member

aajisaka commented Apr 28, 2021

Merged. Thank you @smengcl

@smengcl
Copy link
Contributor Author

smengcl commented Apr 28, 2021 •
edited
Loading

@aajisaka Thanks for reviewing and merging this.

kiran-maturi pushed a commit to kiran-maturi/hadoop that referenced this pull request Nov 24, 2021
@smengcl
YARN-10747. Bump YARN CSI protobuf version to 3.7.1 (apache#2946)
2016dfb 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>

Change-Id: Ib2a4bf775a5804641a91c6f9a486772c9d87c17e
aajisaka pushed a commit that referenced this pull request Mar 22, 2022
@smengcl
YARN-10747. Bump YARN CSI protobuf version to 3.7.1 (#2946)
4153c16 
Signed-off-by: Akira Ajisaka <aajisaka@apache.org>

Change-Id: Ib2a4bf775a5804641a91c6f9a486772c9d87c17e
(cherry picked from commit 5b09ba7)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@aajisaka aajisaka aajisaka approved these changes

@sunilgovind sunilgovind Awaiting requested review from sunilgovind

Assignees

@smengcl smengcl

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@smengcl @aajisaka @hadoop-yetus